Add integration test for restore of file-based programs #8470

RikkiGibson · 2025-07-29T22:41:46Z

Depends on changes in dotnet/roslyn#79669

src/lsptoolshost/server/languageServerEvents.ts

…ion-test-restore

RikkiGibson · 2025-08-06T22:41:26Z

I gotta say, I am not the most thrilled about certain aspects of javascript debugging. (when the "current statement" being debugged is a whole screenful and then some..)

dibarbet · 2025-08-06T23:47:13Z

I gotta say, I am not the most thrilled about certain aspects of javascript debugging. (when the "current statement" being debugged is a whole screenful and then some..)

Depending on what you are debugging that may be expected or not. If you are debugging the extension code running in the tests, there should be source maps which let you step through the typescript. But if you're debugging VSCode itself, unfortunately thats what we have without a lot more setup.

…ion-test-restore

…b.com/dotnet/vscode-csharp into dev/rigibson/integration-test-restore

…ion-test-restore

RikkiGibson · 2025-08-12T17:17:42Z

@dibarbet pointed out the integration test may be failing because the waiter needs to be explicitly enabled in this context.

The test does pass locally, but, maybe lack of the env var was causing a problem in CI. We will find out.

It looks like it would also be an option to add a command line argument to LanguageServer and call AsynchronousOperationListenerProvider.Enable(true);. But, setting the env var seems a bit easier.

RikkiGibson · 2025-08-12T20:35:51Z

This is the problem

2025-08-12 19:10:05.471 [debug] [.NET Restore] /__w/1/s/test/lsptoolshost/integrationTests/testAssets/slnWithCsproj/src/scripts/app1.cs(1,1): error MSB4025: The project file could not be loaded. Data at the root level is invalid. Line 1, position 1.
2025-08-12 19:10:05.512 [debug] [.NET Restore] Failed to run restore on /__w/1/s/test/lsptoolshost/integrationTests/testAssets/slnWithCsproj/src/scripts/app1.cs
2025-08-12 19:10:05.513 [debug] [.NET Restore] Restore complete

This probably means an SDK update is needed.

…ation-test-restore

RikkiGibson · 2025-10-29T03:03:40Z

test/lsptoolshost/integrationTests/fileBasedPrograms.integration.test.ts


-const doRunSuite = process.env.RoslynSkipTestFileBasedPrograms !== 'true';
-console.log(`process.env.RoslynSkipTestFileBasedPrograms: ${process.env.RoslynSkipTestFileBasedPrograms}`);
+const doRunSuite = process.env['ROSLYN_SKIP_TEST_FILE_BASED_PROGRAMS'] !== 'true';


https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#environment-variables

System and user-defined variables (except secret variables) also get injected as environment variables for your platform. When variables convert into environment variables, variable names become uppercase, and periods turn into underscores. For example, the variable name any.variable becomes the variable name $ANY_VARIABLE.

Linux env vars are case sensitive, so, the env. lookup is case sensitive. Since we were not using an uppercase name to access the env var, we weren't seeing it in the test.

Figuring this out took an unreasonable amount of time.

RikkiGibson · 2025-10-29T03:05:55Z

Mac failure:

FAIL test/lsptoolshost/integrationTests/fileBasedPrograms.integration.test.ts (28.106 s)
  ● File-based Programs Tests › Inserting package directive triggers a restore

    Polling did not succeed within the alotted duration: Error: expect(received).toContain(expected) // indexOf

    Expected value: "Newtonsoft"
[main 2025-10-29T01:31:55.708Z] Extension host with pid 15201 exited with code: 1, signal: unknown.
    Received array: ["Microsoft", "static", "System", "unsafe"]

      270 |     }
      271 |
    > 272 |     throw new Error(`Polling did not succeed within the alotted duration: ${error}`);
          |           ^
      273 | }
      274 |
      275 | export async function sleep(ms = 0) {

      at waitForExpectedResult (integrationHelpers.ts:272:11)
      at Object.<anonymous> (fileBasedPrograms.integration.test.ts:55:9)

Maybe the remaining failing legs are using too old of an SDK? They may need to also have the env var added.

RikkiGibson · 2025-11-03T18:07:24Z

@JoeRobich @dibarbet This is ready for review

dibarbet · 2025-11-03T18:12:03Z

test/lsptoolshost/integrationTests/fileBasedPrograms.integration.test.ts

+
+const doRunSuite = process.env['ROSLYN_SKIP_TEST_FILE_BASED_PROGRAMS'] !== 'true';
+console.log(`process.env.ROSLYN_SKIP_TEST_FILE_BASED_PROGRAMS: ${process.env.ROSLYN_SKIP_TEST_FILE_BASED_PROGRAMS}`);
+console.log(`doRunSuite: ${doRunSuite}`);


consider extracting all this to a helper function similar to

vscode-csharp/test/lsptoolshost/integrationTests/integrationHelpers.ts

Line 272 in 65e5e08

export const describeIfCSharp = describeIf(!usingDevKit());

dibarbet · 2025-11-03T18:17:29Z

test/lsptoolshost/integrationTests/fileBasedPrograms.integration.test.ts

+    let exports: CSharpExtensionExports;
+
+    beforeAll(async () => {
+        process.env.RoslynWaiterEnabled = 'true';


Thinking it would be reasonable to set this for all Roslyn integration test runs, even if we don't use it yet.
e.g.

vscode-csharp/test/lsptoolshost/integrationTests/index.ts

Line 10 in 65e5e08

process.env.RUNNING_INTEGRATION_TESTS = 'true';

I can look at that in a followup though myself.

test/lsptoolshost/integrationTests/fileBasedPrograms.integration.test.ts

dibarbet · 2025-11-03T18:20:06Z

azure-pipelines.yml

  displayName: Test Windows
  dependsOn: []
+  variables:
+    ROSLYN_SKIP_TEST_FILE_BASED_PROGRAMS: 'true'


I assume we can enable the tests for windows/mac when we upgrade them to running on .NET 10?

dibarbet · 2025-11-03T18:20:45Z

test/lsptoolshost/integrationTests/testAssets/slnWithCsproj/src/scripts/app1.cs

@@ -0,0 +1,4 @@
+
+using Newton;


Instead of having this be in the file by default, should we insert it during the test? That way we can re-use the same base test file if we need to for other tests?

RikkiGibson added 2 commits July 29, 2025 13:57

add test resource

bd19287

WIP: Integration test for file-based programs restore scenario

1305e21

RikkiGibson requested a review from a team as a code owner July 29, 2025 22:41

RikkiGibson commented Jul 29, 2025

View reviewed changes

src/lsptoolshost/server/languageServerEvents.ts Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

RikkiGibson marked this pull request as draft July 30, 2025 00:56

RikkiGibson added 4 commits July 31, 2025 13:31

Merge remote-tracking branch 'origin/main' into dev/rigibson/integrat…

e0dea11

…ion-test-restore

add D.B.props

3458d52

cleanup

d4b5997

inspect details of notifications from server

f9be157

RikkiGibson mentioned this pull request Jul 31, 2025

Add waitForAsyncOperations LSP request dotnet/roslyn#79669

Merged

RikkiGibson added 10 commits August 6, 2025 19:09

use 'waitForAllAsyncOperationsAsync'

5fd1d9c

simplify

45e4d45

fix lint errors

fa315bc

Merge remote-tracking branch 'origin/main' into dev/rigibson/integrat…

02e435c

…ion-test-restore

WIP: bump roslyn version

0ef2931

rekick CI after adding package to feeds

5a32bb2

fix lint errors

ad92e62

Merge branch 'dev/rigibson/integration-test-restore' of https://githu…

2239641

…b.com/dotnet/vscode-csharp into dev/rigibson/integration-test-restore

Merge remote-tracking branch 'origin/main' into dev/rigibson/integrat…

e4e236a

…ion-test-restore

enable waiter in restore tests via env var

2fd2774

I installed eslint finally

3beee8e

RikkiGibson added 3 commits August 12, 2025 14:07

condition test on new enough sdk

3054919

includePreviewVersions

94a2175

use exact version

386f63e

RikkiGibson added 12 commits October 28, 2025 14:48

Merge remote-tracking branch 'upstream/main' into dev/rigibson/integr…

eb24923

…ation-test-restore

fix yml. fix skip test variable.

3c5d899

fix container name

77781dc

fix for I bet the last time

cc3d74f

Move env var in pipeline

9d742a3

try to inspect what is happening in CI

7cf3de2

try different name

78c2213

use template parameter

3e60d31

fix name

c2f35aa

try to deal with azdo env var weirdness

dd46cce

finish rename

b78119a

rename test file

0baa5ab

RikkiGibson commented Oct 29, 2025

View reviewed changes

RikkiGibson added 3 commits October 29, 2025 12:17

More skips

885011d

Merge branch 'main' into dev/rigibson/integration-test-restore

a56489b

Merge branch 'main' into dev/rigibson/integration-test-restore

c030821

RikkiGibson marked this pull request as ready for review October 30, 2025 17:20

dotnet deleted a comment from azure-pipelines bot Nov 2, 2025

JoeRobich approved these changes Nov 3, 2025

View reviewed changes

dibarbet approved these changes Nov 3, 2025

View reviewed changes

RikkiGibson added 4 commits November 3, 2025 15:05

try to remove sleep(1)

b9eb6a9

edit scripts/app1.cs

3925c0c

extract describeIf helper

293cf59

Merge branch 'main' into dev/rigibson/integration-test-restore

c7ceaf1

RikkiGibson enabled auto-merge (squash) November 4, 2025 21:24

RikkiGibson merged commit 01dd584 into main Nov 4, 2025
33 checks passed

RikkiGibson deleted the dev/rigibson/integration-test-restore branch November 4, 2025 21:51

Add integration test for restore of file-based programs #8470

Add integration test for restore of file-based programs #8470

Uh oh!

Conversation

RikkiGibson commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

RikkiGibson commented Aug 6, 2025

Uh oh!

dibarbet commented Aug 6, 2025

Uh oh!

RikkiGibson commented Aug 12, 2025

Uh oh!

RikkiGibson commented Aug 12, 2025

Uh oh!

RikkiGibson Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

RikkiGibson commented Oct 29, 2025

Uh oh!

RikkiGibson commented Nov 3, 2025

Uh oh!

dibarbet Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

dibarbet Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dibarbet Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

dibarbet Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

RikkiGibson commented Jul 29, 2025 •

edited

Loading